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Abstract 


Test  data  generated  according  to  two  different  multidimensional  item 
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be  detected  when  the  data  sets  were  constrained  to  be  equivalent  in  terms  of  item 
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differences  decreased  as  the  correlation  between  examinee  abilities  increased. 
Furthermore,  these  item  differences  were  small  in  magnitude  and  could  be 
considered  unimportant  or  insignificant  from  a  practical  standpoint.  No 
differences  were  found  at  the  total  test  score  level,  and  it  was  concluded  that,  at 
least  for  the  data  used  in  this  study,  the  models  were  indistinguishable. 
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Comparison  of  Two  Logistic  Multidimensional  Item 
Response  Theory  Models 


Psychometricians  who  have  some  interest  in  multidimensional  item 
response  theory  (MIRT)  modeling  may  be  familiar  with  the  terms,  compensatory 
and  noncompensatory  as  they  relate  to  two  general  model  classification  schemes. 
Ansley  and  Forsyth  (1985)  contrasted  the  two  types  of  model  classifications  as 
follows.  "Compensatory  models,  unlike  noncompensatory  models,  permit  high 
ability  on  one  dimension  to  compensate  for  low  ability  on  another  dimension  in 
terms  of  probability  of  correct  response.  In  the  noncompensatory  models,  the 
minimum  factor  (probability)  in  the  denominator  is  the  upper  bound  for  the 
probability  of  a  correct  response.  Thus,  for  a  two-dimensional  item,  a  person  with 
a  very  low  ability  on  one  dimension  and  very  high  ability  on  the  other  has  a  very 
low  probability  of  correctly  answering  the  item"  (p.  40). 

Typically,  MIRT  models  of  the  compensatory  type,  such  as  the  logistic 
MIRT  model  (Doody-Bogan  &  Yen,  1983;  Hattie,  1981;  Reckase,  1985,  1986)  or 
the  normal  ogive  MIRT  model  (Samejima,  1974)  imply  linear  combinations  of  the 
multidimensional  abilities  in  the  exponent  of  the  expression  for  the  probability  of 
a  correct  response.  In  this  linear  fashion,  a  low  ability  on  one  or  more  of  the  k 
ability  dimensions  can  be  compensated  by  a  higher  ability  on  one  or  more  of  the 
remaining  dimensions.  Because  the  compensation  is  a  characteristic  of  this  linear 
combination,  such  models  are  probably  more  accurately  labeled  linear  MIRT 
models.  A  typical  linear  logistic  MIRT  model  of  the  compensatory  type  can  be 
written  as 
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where 

Cj  =  the  pseudo-guessing  parameter  of  the  y'th  item, 
aim  =  the  discrimination  parameter  for  the  ;'th  item  on 
the  mth  dimension, 

d j  =  the  difficulty  parameter  for  the  j th  item,  and 
0im  =  the  mth  element  in  the  /th  person’s  ability  vector, 

0, 

In  this  model  the  favorable  response  probability,  Pj(Oj),  is  bounded  from 
below  by  cr  However,  because  the  upper  bound  of  /yB;)  is  not  a  function  of  any 

k 

one  ability  dimension,  it  increases  monotonically  as  s  f  increases. 

m-1  'Jm 

On  the  other  hand,  noncompensatory  MIRT  models  (Sympson,  1978; 
Embretson,  1984)  describe  the  probability  of  a  favorable  response  in  terms  of  a 
product  of  k  functions  of  ability  on  a  single  dimension  and  item  characteristics.  In 
its  most  common  form,  a  logistic  MIRT  model  of  this  noncompensatory  or 
multiplicative  type  can  be  written  as 


-  cj  +  0  -  c)  n 
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where  now  we  let  fijm  =  [ajm  (0jm  -  with  b)m  =  the  difficulty  parameter  for  the 
;'th  item  on  the  mth  dimension.  Pj(0,)  is  bounded  by  an  upper  asymptote  equal  to 
the  minimum  of  exp{fjjm}/(l  +  exp{fijm}),  and  the  lower  asymptote,  Cj  for  any  given 
examinee  with  8  =  0j.  Thus,  the  noncompensatory  nature  of  the  model  is  due  to 
the  fact  that  Pj(0,)  can  never  be  greater  than  the  minimum  value  of  the  terms  in 
the  product,  exp{fijm}/(l +exp{fjjm}),  a  function  of  the  smallest  value  of  the  k 


3 


ability  dimensions  for  a  given  examinee.  Because  of  its  multiplicative  form,  the 
model  ta  more  generally  labeled  as  a  multiplicative  MIRT  model. 

Researchers  have  used  the  multiplicative  MIRT  model  to  examine 
characteristics  of  unidimensional  item  response  theory  parameter  estimates 
derived  from  MIRT  response  data  (Ansley  &  Forsyth,  1985)  and  to  model  certain 
multicomponent  latent  traits  in  response  processes  (Embretson,  1984).  Reckase 
(1985)  has  used  a  linear  MIRT  model  on  real  response  data  to  estimate  two- 
dimensional  item  and  person  parameters  on  an  ACT  Assessment  Mathematics 
Usage  test.  However,  no  one  has  actually  shown  that  one  model  is  more 
representative  of  the  actual  item-examinee  response  process  than  the  other.  It 
may  even  be  possible  that  one  model  may  be  appropriate  under  one  set  of 
circumstances  while  the  other  type  may  be  more  appropriate  in  other  situations. 

In  this  paper  we  investigate  the  differences  between  item  responses 
generated  by  these  two  logistic  MIRT  models.  We  have  been  interested  in 
determining  whether  or  not  it  is  possible  to  distinguish  one  model  or  process  from 
the  other  through  some  evaluation  of  response  data.  More  specifically,  our 
concern  has  been  in  establishing  whether  or  not  it  is  possible  to  detect  differences 
between  these  two  MIRT  models,  either  at  the  item  response  or  test  score  level, 
when  the  item  parameters  from  each  MIRT  model  have  been  matched  or  equated 
in  some  sense. 

The  first  task  was  to  establish  the  item  parameters  from  one  of  the  logistic 
MIRT  models  that  would  produce  "reasonable"  p-values  or  proportion-correct 
indices  for  a  specified  examinee  population.  Therefore,  a  target  distribution  of  p- 
values  for  a  20-item  test  was  conceived  and  item  parameters  for  a  linear  or 
compensatory  MIRT  model  were  chosen,  basically  by  trial-and-error,  until  the 
expected  p -value  with  respect  to  this  examinee  population  matched  the  target 
distribution.  Table  1  gives  the  set  of  item  parameters  for  the  20  items  for  the 
model  given  by  equation  (1).  The  table  also  gives  the  expected  value  of  each  p- 
value  under  the  assumption  that  the  ability  vector,  0,  for  the  examinee  population, 
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was  distributed  as  bivariate  normal  with  mean  vector,  0,  and  valance-covariance 
matrix  of  ones  along  the  diagonal  and  with  nondiagonal  values  equal  to  rho 
(.00,  .25,  .50,  or  .75).  All  c-parameters  were  set  to  zero. 


Insert  Table  1  Here 


In  order  to  produce  a  comparable  or  "matched"  set  of  noncompensatory,  or 
multiplicative  model  item  parameters,  estimates  of  these  item  parameters  were 
obtained  by  minimizing 

zi  [  fye,  a,  d)  ]-[  VC*,  <)  ]>  <3> 

i-1 


for  N  =  2000  randomly  selected  examinees  with  ability,  0,  distributed  as  given 
previously,  where  Pc  and  PNC  represent  logistic  MIRT  models  given  by  equations 
(1)  and  (2),  respectively.  This  process  was  repeated  for  10  replications  for  each  of 
k  -  1,  2,  ...,  20  items  to  insure  that  the  estimates  obtaineH  weren’t  unduly 
influenced  by  the  samples  selected  or  the  starting  values  used.  Mean  values  of 
the  replication  estimates  yielded  the  noncompensatory  item  parameters  listed  in 
Tables  2-5,  for  rho  values  of  .00,  .25,  .50,  and  .75.  The  expected  value  of  each 
item’s  />-value  is  given  in  the  last  column  of  each  table.  Because  the  least  squares 
minimization  procedure  produces  unbiased  estimates  of  PNC,  the  expected  value  of 
each  p-value  under  the  noncompensatory  model  should  be  equal  to  that  of  the 
compensatory  model,  within  some  estimation  error.  Equivalence  of  /^-values  was 
the  critical  matching  criterion  between  the  two  MIRT  models. 

Insert  Tables  2-5 
Here 
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Model  Differences  at  the  Total  Test  Level 

By  treating  the  two  sets  of  item  parameters  as  known  for  each  of  the  two 
MIRT  models,  we  first  investigated  the  differences  between  expected  number- 
correct  score  frequencies  of  a  20-item  test  when  0  was  distributed  as  a  bivariate 
normal  random  vector  with  distributions  given  previously.  These  frequencies  were 
estimated  by  evaluating  either  the  number-correct  distribution  under  the 
compensatory  model,  hc(y)  or  the  noncompensatory  model,  *nc(>’X  for  y 
=  0,  1,  2,  ...  ,20,  or 


My) 


®  <0 

'  |  /c(y  I  0)  8(B)  de,  de2 

-®  -® 


(4) 


and 


VW  -  J  |  /NC(v  I  9)  S(9)  M, 


(5) 


In  each  case,  the  conditional  frequencies,  f^(y  |  0)  and  /  (y  I  0)  ,  were 
computed  using  either  models  (1)  or  (2),  and  a  recursive  procedure  described  by 
Lord  and  Wingersky  (1984).  Table  6  gives  the  signed  differences  between  the 
frequencies,  hc(y)  -  hsc(y),  fory  =  0,  1,  2  ,  ...,  20,  for  rho  values  of  .00,  .25,  .50, 
and  .75.  The  greatest  differences,  as  expected,  occurred  for  the  highest  number- 
correct  scores,  but  the  differences  in  frequencies  were  small,  never  greater  than 
.015.  For  most  number-correct  score  values,  these  differences  became  smaller  as 
rho  increased. 
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Another  way  to  assess  the  significance  of  these  differences  was  to 
determine  how  much  data  would  need  to  be  observed  before  the  differences  were 
statistically  detectable.  This  was  done  by  calculating  the  minimum  sample  size 
required  to  reject  the  homogeneity  of  parallel  populations  with  given  levels  of  test 
significance  and  power.  These  calculations  assumed  a  multivariate  normal 
approximation  for  each  model’s  multinomial  distribution  of  observed-score 
frequencies  which  in  turn  produced  the  quadratic  form  of  the  noncentrality 
parameter  of  a  noncentral  chi  square  distribution.  The  minimum  sample  size 
followed  as  a  direct  function  of  this  parameter,  the  specified  test  significance,  and 
power.  For  example,  with  a  significance  level  of  .01  and  power  equal  to  .95,  the 
minimum  sample  sizes  were  1678,  3242,  7466,  and  15311  for  correlated  ability 
distributions  with  rho  equal  to  .00,  .25,  .50,  and  .75,  respectively.  These  sample 
sizes  state  that  even  in  the  unlikely  event  of  uncorrelated  ability  distributions,  it 
would  still  require  at  least  1678  observed  scores  from  both  the  compensatory  and 
noncompensatory  MIRT  models  before  the  null  hypothesis  of  model  equivalence 
could  be  rejected  with  a  power  of  .95. 

Insert  Table  6  Here 


The  first  four  (central)  moments  of  each  number-correct  distribution  are 
given  in  Table  7  for  each  value  of  rho.  Both  distributions  were  negatively  skewed 
with  the  compensatory  distribution  slightly  more  platykurtic  and  both  were 
generally  flatter  than  the  normal  distribution.  The  variances  of  the  number- 
correct  scores  increased  with  an  increase  in  rho,  and  in  general,  the  distributions 
of  number-correct  scores  became  increasingly  similar  as  rho  increased. 


Insert  Table  7  Here 
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A  contour  plot  of  the  (signed)  difference  between  the  number-correct  true 
scores  under  the  two  models,  or 

20  20 

SP.  (0)  -  SP,  (6) 

H  Jc  ,-i  1N( 

was  another  way  to  observe  model  differences  at  the  total  test  level  for  various 
(0,,  02)  points  in  the  ability  space.  The  greatest  differences  occurred  when  either 
0,  or  0,  was  low.  See  Figures  1-4  for  rho  values  of  .00.  .25,  .50,  and  .75, 
respectively.  It  should  be  noted  that,  in  these  plots,  the  only  influence  of  rho  was 
through  the  values  of  the  noncompensatory  item  parameters.  Recall  that  the 
compensatory  item  parameters  were  fixed  for  all  values  of  rho.  Therefore,  when 
interpreting  these  contour  plots,  one  has  to  mentally  superimpose  the  appropriate 
bivariate  normal  distribution  over  the  contours  in  order  to  evaluate  the 
importance  of  the  true-score  differences  observed. 


Insert  Figures  3-4 
Here 


Another  way  to  compare  the  two  MIRT  models  was  to  observe  the  amount 
of  multidimensional  information  (M1NF)  for  different  points  in  the  ability  space 
between  the  two  models.  MINF  has  been  defined  (Reckase,  1986)  as  a  direct 
generalization  of  the  unidimensionai  IRT  concept  of  item  information  (i.e,  the 
ratio  of  the  square  of  the  slope  of  the  item  characteristic  curve  at  an  ability  point, 
0,  to  the  variance  of  the  error  of  the  item  score  at  that  level  of  0).  For  the 
definition  of  MINF,  the  slope  of  the  item  characteristic  surface  must  be  evaluated 
in  a  particular  direction,  a,  a  vector  of  angles  with  the  coordinate  axes  of  the 
ability  space. 

Plots  of  the  absolute  difference  between  the  compensatory  and 
noncompensatory  test  information  vectors  (i.e,  the  sum  of  item  information  across 
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the  20  items)  for  item  rameters  estimated  with  rho  values  of  .00,  .25,  .50,  and 
.75  (Figures  5-8,  respectively)  showed  that  model  differences  might  be  significant 
if  abilities  were  negatively  correlated.  However,  for  all  "likely"  ability 
distributions,  there  were  no  meaningful  differences  in  MINF  between  the  two 
models,  and  these  absolute  differences  appeared  to  decrease  as  rho  increased. 


Insert  Figures  5-8  Here 


Model  Differences  at  the  Item  Level 

It  was  also  of  interest  to  evaluate  the  differences  between  models  at  the 
single  item  response  level.  There  were  two  ways  in  which  this  was  done.  The 
first  involved  the  evaluation  of  the  ideal  observer  index  (Davey,  Levine,  & 

Williams,  1989;  Levine,  Drasgow,  Williams,  McCusker,  &  Thomasson,  1990).  A 
more  complete  definition  of  this  index  is  provided  in  the  appendix  of  this  paper. 
However,  a  simplified  definition  is  as  follows.  The  ideal  observer  index  (IOI)  is  a 
measure  of  the  proportional  number  of  times  that  a  correct  decision  is  made 
concerning  which  of  the  two  competing  models  produced  a  particular  response  to 
an  item.  The  decision  is  one  that  is  made  hypothetically  by  an  "ideal  observer,"  or 
an  individual  who  has  access  to  all  of  the  information  necessary  to  yield  the 
highest  possible  percent  of  model  classification  (i.e.,  compensatory  vs. 
noncompensatory).  As  far  as  the  ideal  observer  is  concerned,  if  the  item  response 
data  fail  to  distinguish  between  the  two  competing  models,  then  the  value  of  this 
index  would  be  at  or  near  the  chance  level  of  .5.  Conversely,  readily 
distinguishable  models  should  yield  an  index  near  1.0. 

Table  8  shows  that  the  IOI  was  greater  than  chance,  implying  that  there 
was  a  difference  between  the  models  for  all  20  items.  However,  the  IOI  was 
never  greater  than  .60  and  was  greater  than  .55  for  only  three  items,  numbers  3,  6, 
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and  7,  when  rho  was  .00.  The  value  of  the  IOI  decreased  for  each  item  as  rho 
increased,  implying  that  it  became  more  difficult  to  distinguish  between  the 
models  as  the  correlation  coefficient  increased. 

One  way  to  think  of  the  magnitude  of  the  IOI  was  to  imagine  how  many 
trials  of  the  IO  experiment  would  be  necessary  before  the  ideal  observer  could 
ascertain,  with  some  given  level  of  certainty,  that  the  models  were  actually 
distinguishable.  This  would  be  comparable  to  a  test  of  the  difference  between  any 
obtained  IOI  from  Table  8  and  the  null  proportion  of  correct  model  classifications 
due  to  chance.  For  example,  to  be  able  to  detect  a  true  difference  between  the 
models  for  item  number  6  with  a  zero  value  of  rho  would  require  at  least  40  trials 
of  the  IO  experiment.  This  would  be  comparable  to  a  test  of  the  null  proportion 
of  correct  classifications  due  to  chance  or  .50  versus  the  (true)  alternative 
proportion  (.555)  with  a  significance  of  .01  and  power  of  .95.  Conversely,  a  true 
IOI  of  .52  would  require  more  than  290  trials  at  similar  levels  of  test  significance 
and  power. 


Insert  Table  8  Here 


Another  way  to  evaluate  model  differences  at  the  item  level  was  to  use  a 
generalized  MIRT  model,  or  a  reparameterization  of  both  the  compensatory  and 
noncompensatory  models  into  a  single  MIRT  model,  or 


+  O-Cj) 


efijrfij2 

1  +  ef"r(>>2  + 


(6) 


where  m  represented  an  indicator  variable  such  that 


M  = 


0,  for  the  linear  or  compensatory  MIRT  model, 

1,  for  the  multiplicative  or  noncompensatory  MIRT  model. 
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Item  response  data,  X;j,  were  generated  from  samples  of  size  1 V  =  2000  of 
0j  drawn  from  the  bivariate  normal  distributions  mentioned  previously.  The 
response  data  were  known  to  have  been  produced  by  either  the  compensatory  or 
noncompensatory  MIRT  model  and  were  simulated  by  comparing  the  known 
values  of  Pfo)  to  a  pseudorandomly  drawn  uniform  deviate,  «,  such  that 


1,  0  <  w  <  />(«,) 

0,  P j(0j)  <  a)  <  1  . 


The  least  squares  estimation  procedure  was  used  to  estimate  the 
generalized  MIRT  model  parameters.  Each  estimation  was  replicated  10  times 
with  randomly  selected  starting  values.  Either  four  or  five  unique  item 
parameters  were  estimated  from  the  generalized  MIRT  model,  as  given  by 
equation  (6).  The  same  item  parameters  that  were  given  in  tables  1-5  were  used 
to  generate  the  response  data  for  the  estimation  procedure.  When  the  response 
data  were  generated  by  the  compensatory  model,  a,,  a2,  and  d  (i.e.,  d  =  -  axbx  - 
ap2)  as  well  as  p,  were  estimated.  When  the  response  data  were  generated  by 
the  noncompensatory  model,  ax,  a2,  bx,  b2,  and  m  were  estimated. 

Table  9  shows  the  average  bias  in  the  item  parameter  estimates  and  the 
standard  deviations  of  the  estimates  (in  parentheses).  For  compensatory  data,  the 
model  parameter,  m  was  estimated  fairly  accurately  for  the  uncorrelated  situation, 
but  the  amount  of  bias  and  the  standard  deviation  of  the  estimates  increased  as 
rho  increased.  A  similar  situation  occurred  with  noncompensatory  data. 

However,  although  the  amount  of  estimation  error  increased  as  the  correlation 
between  the  abilities  increased,  the  model  still  remained  identifiable ,  in  the  sense 
that  for  compensatory  data,  the  n  estimates  were  statistically  "close"  to  zero. 
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Likewise,  for  noncompensatory  data,  the  m  estimates  were  satistically  "close"  to 
one. 

Insert  Table  9  Here 


The  IOI  analysis  and  the  generalized  MIRT  model  estimation  gave  similar 
results.  That  is,  there  were  model  differences  at  the  item  level,  but  these 
differences  tended  to  decrease  as  the  correlation  in  abilities  increased.  The 
generalized  MIRT  analysis  also  suggested  that  these  differences  might  still  be 
estimable,  however,  even  when  abilities  are  strongly  correlated. 

Summary'  and  Conclusions 

These  analyses  and  results  seem  to  indicate  that  even  though  it  is  difficult 
to  observe  model  differences  at  the  overall  test  score  level,  there  still  may  be 
measurable  differences  between  the  responses  at  the  item  level.  Because  the 
matching  criterion  between  the  two  models  resulted  in  similar  expected  /7-values, 
we  anticipated  small  differences  at  the  total  test  score  response  level,  or  at  the 
true  score  level.  The  differences  that  were  detected  at  this  level  were  consistent 
with  the  differences  implied  in  the  two  models.  Fewer  high,  number-correct 
scores  or  estimated  true  scores  were  observed  from  the  noncompensatory  model, 
but  these  and  other  total  test  differences  decreased  as  rho  increased.  As  for  the 
item  response  level  analysis,  both  the  IOI  and  the  generalized  MIRT  model 
estimation  showed  that  it  is  possible  to  quantify  these  differences  and  to 
distinguish  between  the  data  generated  by  carefully  matched  item  response 
models  of  these  two  types.  However,  these  differences,  although  real,  are  very 
small  and  probably  not  significant  from  any  practical  standpoint. 
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Although  it  is  difficult  to  generalize  beyond  the  two-dimensional  situation 
used  in  the  present  study,  it  would  appear  to  be  difficult  to  distinguish  between 
the  two  models  without  the  benefit  of  any  prior  knowledge  of  item  parameters  or 
abilities.  Even  with  such  prior  knowledge,  response  data  generated  by  the  models 
are  nearly  indistinguishable,  especially  with  correlated  abilities,  which  is  likely  the 
case  in  many  real  testing  situations. 
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Appendix 

Analytical  Definition  of  the  Ideal  Observer  Irulex 

A  hypothetical  observer  is  presented  with  two  abilities,  tt  and  t2,  each  with 
their  associated  item  responses,  U!  and  u2.  The  observer  is  informed  that  one 
ability-response  pair  was  generated  by  one  of  two  competing  item  response 
models,  while  the  other  pair  was  generated  under  the  second  model.  The  task  is 
to  correctly  match  each  ability-response  pair  with  the  proper  generating  model. 
To  make  this  decision,  the  observer  is  given  access  to  both  competing  item 
response  functions,  P1  and  P2 ,  and  the  common  ability  distribution,  f(t). 

An  ideal  observer  bases  this  decision  on  an  optimal  rule,  6,  which  is 
determined  by  the  ratio  of  likelihood  functions,  Lj(tj,Uj)  =  Pj(tj)Uj  Qj(tj)1  Uj  ,  where 
Q^tj)  =  1  -  Pi(tj),  i  =  1,  2;  j  =  1,  2.  The  decision  rule,  S,  is  then  defined  as 

if  L^tj.u^-Ljftj.uj)  >  L1(t2,u2)*L2(t],u1),  then  decide  model  {Pa;f} 
produced  sample  {t^uj  while  model  {P2;f}  produced  {t2,u2}. 

S  = 

if  Lj^u^i^tj.Uj)  >  L1(t„u1)-L2(t2,u2),  then  decide  model  {P2;f} 
produced  sample  {t^uj  while  model  {Pj;f}  produced  {t2,u2}. 

The  probability  of  this  decision  rule  being  correct,  given  the  model,  is 


Prob[5  correct |  model]  =  ProbfL^tpUj)  -L^Uj)  >  Lj(t2,u2)  •  L^tj.Uj)!  {Pj;f}&{P2;f}]  + 

Prob[L!(t2,u2)  -L^u,)  >  Lj^u,)  •  L^Uj)  |  {P2;f}&{Pj;f}]. 
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The  response  pair,  u,  where  u  =  (uj,u2),  can  be  defined  in  four  possible 
patterns:  (1,1),  (1,0),  (0,1),  and  (0,0).  Therefore, 

Prob[L1(t1,u1)  -L,(t2,u2)  >  Lj(t2,u2)  *  L2(t1,ul)  |  {P^&IP^f}]  = 

ProblP,^)  *P2(t2)  >  P,(t2)  •P2(t,)|u  =  (l,l)]  •Prob[u  =  (l,l)|{P1;f}&{P2;f}] 

+  ProbtPjO,)  * 02(t2)  >  Q,(t2)  * P2(tx) | u  =  (1,0)]  -Prob[u  =  (l,0)|{Pi;f}&{P2;f}] 

+  ProblO.O,)  -P2(t2)  >  Pa(t2)  •O2(t1)|u  =  (0,l)]  •Prob[u  =  (0,l)|{P1;f)&{P2;f}] 

+  ProbtQ^)  *  Q2(t2)  >  Qi(t2)  *Q2(t1)|u  =  (0,0)l  •  Prob[u  =  (0,0) |  {Pi;f}&{P2;f}}. 

Define  ir,  -  f(>)  f(g)  d<  dg- 

Then,  Prob^t^)  *  L^Uj)  >  L^Uj)  *  L^Ui)!  {P^O&jPzjf}]  = 

ttu  ProbtP,^)  *P2(t2)  >  P,(t2)  •P2(t1)|u  =  (l,l)]  + 

*r10  Prob^t,)  *  02(t2)  >  Q,(t2)  •  P2(t,)|u  =  (l,0)]  + 

7rni  ProbfQ^tj)  *P2(t2)  >  Pj(t2)  •Q2(t1)|u  =  (0,l)]  + 

ProblQ^)  *  Q2(t2)  >  Qi(t2)  -Q^)  |  u =(0,0)1. 

Similarly,  ProtyL/t^Uz)  ‘^(tj.u,)  >  L,(t„Ui)  •L2(t2,u2)|{P2;f}&{P1;f)l  = 
Prob[Pj(t2)  -Pjto)  >  Pj(tj)  -  P2(t2)|u  =  (l,l)]  *  Prob[u  =  (1,1)  1  {P2;f}&{P,;f}] 

+  Prob[Pj(t2)  *  Q2(tj)  >  Qfa)  *  P 2(t2)  I  u  =  ( 1,0)]  -  Prob[u  =  ( 1 ,0)  |  { P2;f} &  { P, ;f }  ] 

+  Prob[Q,(t2)  -P2(t,)  >  P,(t,)  * Q2(t2) | u  =  (0, 1  )1  •Prob[u  =  (0,l)|{P2;f}&{P1;f)l 
+  Prob[Q,(t2)  -Q2(tj)  >  Q,(t,)  *Q2(t2)|u  =  (0,0)l  •Prob[u  =  (0,0)|{P2;f}&{P,;f}l 
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Then,  Prob[LA,u2)  -LA,^)  >  L^.u,)  •  L2(t2,u2)|  {P^fJ&fP^f}]  = 


nn  Prob[P A)  -PA)  >  PA)  -P2(t2)|u  =  (l,l)]  + 

»io  Prob[PA)  -  QA)  >  Q,(t,)  *P2(t2)|u=(l,0)]  + 
»oi  Prob[Oj(t2)  -  PA)  >  PA)  •O2(t2)|u  =  (0,l)]  + 
jtoo  Prob[Q,(t2)  *  Q2(t,)  >  QA)  *  Q2(t2)|u  =  (0,0)]. 


Let  u  be  defined  as  that  region  of  the  ability  space  where 

PA)1*1  *  QA)"1’1  *  P2(t2)U2  *  QA)02'1  >  PA)02  •  Q1(t2)U2’1  •  P2(tj)ui  •  QA)"1'1 

holds,  and  likewise  let  nu  u  be  defined  as  that  region  of  the  ability  space  where 

PA)°2  •  QA)"2'1  *  P2(t,r  •  QA)"1'1  >  Pi(t,r  •  QA)1*1'1  *  P2(t2)U2  *  QA)02'1 


is  true.  Then 


prob[p,(t,)  •  p2(t2>  >  p,(t2)  •  p2(t,)iu  -  d,i)]  -  r  r  f(t>  f<w  dt  dg. 


Prob[P,(t,)  *  02(t2)  >  Q,(t2)  •  P2(t,)lu  -  (1,0)]  -  JJ  f(t)  f(g)  dt  dg, 

“10 


Prob[Q,(t,)  -P2(t2)  >  P,(tj)  •  Q2(t,)lu  -  (0,1)]  -  JJf(t)  f(g)  dt  dg, 
and 

Prob[Q,(t,)  ■  02(t2)  >  Q,(t2)  *  Q2(t,)  I  u  -  (0,0)]  -  [  [  f(t)  f(g)  dt  dg. 

JKoo 
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Then 


Prob[P,(t2)  -P2(t,)  >  P,(t,)  -P,(t2)l.i  -  (1,1)1  -  JJf(t)  f(g)  dt  dg, 


Prob(P,(t2)  •  Q2(tl)  >  0,(t,)  ■  P,(4)lu  -  (0,1)]  -  JJ  f(t)  f(g)  dt  dg, 

°01 

Prob[Q,(y  -P2(t,)  >  0,(4)  •  P,(t,)lu  -  (1,0)1  -  JJf(t)  f(g)  dt  dg, 

°10 


Prob[Q,(4)  -0,(1,)  >  Q,(t,)  ‘  0,(4)  I  u  -  (0,0)1  -  JJf(t)  f(g)  dt  dg. 


Thus,  Prob[<S  correct  |  model]  = 


*oiJ  f  f(0f(g)dtdg  +  *<»[  f  f(t)f(g)dtdg  + 

^01  HX) 

*nJJ  f(Of(g)dtdg  +  7r10|J  f(t)f(g)dtdg  + 

flll  °oi 

»oiJJ  f(Of(g)dtdg  +  7r00JJ  f(t)f(g)  dtdg 


»n  +  ^ioJ  f  f(t)f(g)dtdg  +  JJ  f(t)f(g)dtdg)  + 

“10  Dot 

^01JJ  f(t)f(g)dtdg  +  JJf(t)f(g)dtdg)  +  xm 

**01  Oio 
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Finally,  Prob[S  correct]  =  Prob[<5  correct  |  model]  •  Prob[selecting  a 
model].  Because  each  model  is  equally  likely,  the  probability  of  selecting  a  model 
is  equal  to  .5.  Thus,  Prob[<5  correct]  =  .5(Prob[6  correct! model]). 
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Table  1 

Original  Item  Parameters  for  the  Compensatory  Model 


E(/?-value) 

rho 


Item  # 

al 

a2 

d 

.00 

.25 

.50 

.75 

01 

0.90 

1.31 

-0.67 

.39 

.39 

.40 

.40 

02 

2.10 

0.50 

-1.13 

.34 

.35 

.36 

.36 

03 

0.89 

1.10 

0.52 

.59 

.59 

.58 

.58 

04 

0.99 

1.00 

-0.44 

.42 

.42 

.43 

.43 

05 

0.58 

1.65 

0.78 

.63 

.62 

.62 

.61 

06 

0.91 

1.27 

0.42 

.57 

.57 

.57 

.56 

07 

1.03 

0.95 

1.08 

.69 

.68 

.67 

.67 

08 

0.32 

2.27 

0.38 

.55 

.55 

.55 

.55 

09 

0.61 

0.72 

1.63 

.80 

.79 

.79 

.78 

10 

0.67 

1.12 

0.60 

.61 

.61 

.60 

.60 

11 

0.91 

0.91 

-0.21 

.46 

.46 

.46 

.47 

12 

0.64 

1.72 

-0.05 

.49 

.49 

.49 

.49 

13 

1.65 

0.38 

0.40 

.57 

.56 

.56 

.56 

14 

0.18 

1.61 

1.84 

.78 

.78 

.78 

.77 

15 

0.82 

1.02 

0.09 

.52 

.52 

52 

.51 

16 

1.45 

0.81 

-0.24 

.46 

.46 

.46 

.46 

17 

1.64 

0.62 

0.85 

XA 

.63 

.63 

.62 

18 

0.77 

0.76 

-0.91 

.32 

.33 

.34 

.34 

19 

1.46 

0.62 

0.10 

.52 

.52 

.52 

.52 

20 

0.39 

1.37 

0.32 

.56 

.56 

.55 

.55 
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Table  2 

Item  Parameters  for  the  Noncompensatory  Model  with  Rho  =  .00 


Item  # 

al 

a2 

b2 

E(p-value) 

01 

1.26 

1.60 

-0.92 

-0.15 

.38 

02 

2.30 

1.04 

0.38 

-2.28 

.34 

03 

1.22 

1.39 

-1.42 

-0.99 

.59 

04 

1.32 

1.35 

-0.62 

-0.58 

.42 

05 

1.02 

1.82 

-2.71 

-0.62 

.62 

06 

1.25 

1.53 

-1.45 

-0.79 

.56 

07 

1.30 

1.26 

-1.48 

-1.63 

.68 

08 

0.92 

2.38 

-3.95 

-0.22 

.55 

09 

0.93 

1.00 

-2.75 

-2.35 

.80 

10 

1.05 

1.37 

-1.96 

-0.90 

.61 

11 

1.24 

1.25 

-0.78 

-0.75 

.46 

12 

1.07 

1.92 

-2.17 

-0.19 

.49 

13 

1.81 

0.88 

-0.36 

-3.25 

.56 

14 

0.85 

1.67 

-5.26 

-1.17 

.78 

15 

1.17 

1.32 

-1.21 

-0.75 

.51 

16 

1.71 

1.23 

-0.27 

-1.35 

.45 

17 

1.83 

1.06 

-0.68 

-2.55 

.63 

18 

1.09 

1.09 

-0.31 

-0.32 

.32 

19 

1.69 

1.07 

-0.35 

-1.98 

.51 

20 

0.88 

1.54 

-2.98 

-0.41 

.55 
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Table  3 

Item  Parameters  for  the  Noncompensatory  Model  with  Rho  ~  .25 


Item  # 

al 

a2 

b2 

E(/?-value) 

01 

1.38 

1.74 

-0.79 

-0.14 

.39 

02 

2.40 

1.14 

0.35 

-1.88 

.34 

03 

1.36 

1.50 

-1.27 

-0.91 

.58 

04 

1.44 

1.45 

-0.56 

-0.51 

.42 

05 

1.17 

1.94 

-2.30 

-0.60 

.61 

06 

1.40 

1.66 

-1.28 

-0.73 

.56 

07 

1.45 

1.40 

-1.34 

-1.47 

.72 

08 

1.05 

2.47 

-3.30 

-0.22 

.55 

09 

1.02 

1.09 

-2.49 

-2.17 

.79 

10 

1.17 

1.47 

-1.72 

-0.85 

.60 

11 

1.34 

1.34 

-0.71 

-0.68 

.46 

12 

1.21 

2.06 

-1.82 

-0.20 

.49 

13 

1.90 

0.98 

-0.36 

-2.80 

.56 

14 

0.93 

1.72 

-4.65 

-1.15 

.78 

15 

1.29 

1.42 

-1.08 

-0.69 

.51 

16 

1.84 

1.33 

-0.27 

-1.16 

.45 

17 

1.97 

1.20 

-0.66 

-2.19 

.62 

18 

1.15 

1.16 

-0.28 

-0.27 

.33 

19 

1.80 

1.18 

-0.35 

-1.71 

.51 

20 

0.98 

1.61 

-2.57 

-0.40 

.55 
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Table  4 

Item  Parameters  for  the  Noncompensatory  Model  with  Rho  =  .50 


Item  # 

ai 

a2 

b, 

b2 

E(/?-value) 

01 

1.52 

1.82 

-0.66 

-0.12 

.39 

02 

2.48 

1.27 

0.32 

-1.51 

.35 

03 

1.49 

1.63 

-1.14 

-0.85 

.58 

04 

1.54 

1.54 

-0.50 

-0.45 

.42 

05 

1.32 

2.04 

-1.97 

-0.59 

.61 

06 

1.55 

1.79 

-1.13 

-0.68 

.56 

07 

1.58 

1.55 

-1.23 

-1.33 

.67 

08 

1.20 

2.51 

-2.78 

-0.22 

.55 

09 

1.10 

1.17 

-2.30 

-2.03 

.78 

10 

1.28 

1.56 

-1.53 

-0.80 

.60 

11 

1.44 

1.43 

-0.64 

-0.61 

.46 

12 

1.36 

2.1? 

-1.54 

-0.19 

.49 

13 

1.96 

1.09 

-0.36 

-2.39 

.56 

14 

1.03 

1.77 

-4.07 

-1.13 

.77 

15 

1.39 

1.51 

-0.97 

-0.64 

.51 

16 

1.95 

1.47 

-0.26 

-0.99 

.46 

17 

2.08 

1.35 

-0.63 

-1.89 

.62 

18 

1.21 

1.20 

-0.23 

-0.23 

.33 

19 

1.89 

1.30 

-0.34 

-1.46 

.51 

20 

1.08 

1.66 

-2.23 

-0.40 

.55 
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Table  5 

Item  Parameters  for  the  Noncompensatory  Model  with  Rho  =  .  75 


Item  # 

ai 

a2 

bi 

b2 

E(/?-value) 

01 

1.65 

1.92 

-0.51 

-0.10 

.40 

02 

2.53 

1.43 

0.31 

-1.14 

.35 

03 

1.60 

1.73 

-1.01 

-0.77 

.58 

04 

1.63 

1.64 

-0.42 

-0.39 

.43 

05 

1.48 

2.14 

-1.67 

-0.57 

.61 

06 

1.69 

1.92 

-0.98 

-0.62 

.56 

07 

1.69 

1.66 

-1.13 

-1.21 

.66 

08 

1.36 

2.57 

-2.25 

-0.22 

.55 

09 

1.15 

1.21 

-2.17 

-1.93 

.78 

10 

1.38 

1.63 

-1.36 

-0.76 

.60 

11 

1.50 

1.51 

-0.56 

-0.54 

.46 

12 

1.53 

2.23 

-1.22 

-0.19 

.49 

13 

1.98 

1.26 

-0.36 

-1.99 

.56 

14 

1.15 

1.78 

-3.60 

-1.11 

.77 

15 

1.47 

1.59 

-0.85 

-0.59 

.51 

16 

2.03 

1.63 

-0.23 

-0.81 

.46 

17 

2.15 

1.53 

-0.61 

-1.60 

.62 

18 

1.24 

1.24 

-0.18 

-0.18 

.34 

19 

1.94 

1.44 

-0.33 

-1.23 

.51 

20 

1.17 

1.70 

-1.92 

-0.40 

.55 
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Table  6 

Compensatory  Minus  Noncompensatory  Density  Differences  in  Number-correct  Score 


Number-correct 
score  (y) 

rho 

.00 

.25 

.50 

.75 

20 

.013 

.014 

.014 

.011 

19 

.015 

.012 

.009 

.004 

18 

.012 

.007 

.003 

.000 

17 

.007 

.003 

.000 

-.002 

16 

.002 

-.001 

-.002 

-.003 

15 

-.003 

-.003 

-.004 

-.003 

14 

-.006 

>q 

s 

i 

-.004 

-.003 

13 

-.009 

-.007 

-.005 

-.003 

12 

-.011 

-.007 

-.005 

-.002 

11 

-.012 

-.008 

-.005 

-.002 

10 

-.012 

-.008 

-.004 

-.001 

9 

-.011 

-.007 

-.004 

-.001 

8 

-.009 

-.006 

-.003 

.000 

7 

-.006 

-.004 

-.002 

.001 

6 

-.003 

-.002 

-.001 

.001 

5 

.001 

.000 

.001 

.002 

4 

.005 

.002 

.002 

.002 

3 

.008 

.005 

.003 

.002 

2 

.009 

.006 

.004 

.001 

1 

.008 

.006 

.003 

-.001 

0 

-.005 

.003 

.001 

-.004 
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Table  7 

Central  Moments  of  Number-correct  Scores 


MIRT  Models 

rho 

Mean 

Second 

Central 

Moment 

Third 

Central 

Moment 

Fourth 

Central 

Moment 

.00 

10.90 

25.79 

-16.56 

1362.83 

Compensatory 

.25 

10.88 

29.40 

-20.44 

1680.36 

.50 

10.86 

32.64 

-24.01 

1980.03 

.75 

10.84 

35.57 

-27.27 

2262.98 

.00 

10.79 

20.67 

-9.42 

946.49 

Noncompensatory 

.25 

10.78 

25.43 

-15.86 

1336.75 

.50 

10.78 

30.12 

-24.30 

1760.64 

.75 


10.78 


34.70 


-32.74 


2200.57 
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Table  8 

Ideal  Observer  Index 


rho 


Item  # 

.00 

.25 

.50 

.75 

01 

.5479 

.5397 

.5295 

.5179 

02 

.5311 

.5265 

.5205 

.5128 

03 

.5513 

.5418 

.5307 

.5183 

04 

.5461 

.5377 

.5279 

.5171 

05 

.5421 

.5353 

.5265 

.5157 

06 

.5550 

.5451 

.5332 

.5194 

07 

.5511 

.5419 

.5304 

.5175 

08 

.5243 

.5212 

.5165 

.5102 

09 

.5276 

.5227 

.5162 

.5092 

10 

.5430 

.5351 

.5254 

.5149 

11 

.5435 

.5355 

.5260 

.5156 

12 

.5448 

.5375 

.5281 

.5166 

13 

.f:.9i 

.5246 

.5185 

.5112 

14 

.5124 

.5109 

.5082 

.5048 

15 

.5456 

.5370 

.5271 

.5161 

16 

.5497 

.5411 

.5307 

.5182 

17 

.5442 

.5371 

.5276 

.5162 

18 

.5281 

.5232 

.5175 

.5114 

19 

.5425 

.5352 

.5260 

.5156 

20 

.5292 

.5241 

.5179 

.5108 
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Table  9 

Average  Bias  ( parameter  estimate  -  true  parameter)  and  Standard  Deviation 
of  Bias  in  Estimates  of  the  Generalized  MIRT  Model  Parameters 


Response 

Data  Model 

rho 

al 

a2 

d 

bi 

b2 

M 

.00 

.044 

(.042) 

.024 

(.073) 

.069 

(.158) 

.013 

(.009) 

Compensatory 

.25 

.044 

(.047) 

.040 

(.042) 

.125 

(.275) 

.026 

(.052) 

.50 

.078 

(.055) 

.069 

(.081) 

.255 

(.238) 

.064 

(.060) 

.75 

.098 

(.128) 

.113 

(.080) 

.787 

(1.930) 

.107 

(.094) 

.00 

-.008 

(.099) 

.009 

(.115) 

.130 

(.448) 

.230 

(.354) 

-.199 

(.163) 

Noncompensatory 

.25 

-.006 

(.090) 

-.004 

(.083) 

.250 

(.622) 

.254 

(.464) 

-.197 

(.144) 

.50 

.039 

(.145) 

-.076 

(.104) 

.191 

(.888) 

.183 

(.265) 

-.200 

(.125) 

.75 

-.155 

(.220) 

-.059 

(.105) 

.071 

(.439) 

.250 

(.421) 

-.288 

(.175) 

Note:  standard  deviations  are  in  parentheses 
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Figure  Captions 

Figure  1.  Difference  Between  Compensatory  and  Noncompensatory  True  Scores: 
Rho  =  .00 

Figure  2.  Difference  Between  Compensatory  and  Noncompensatory  True  Scores: 
Rho  =  .25 

Figure  3.  Difference  Between  Compensatory  and  Noncompensatory  True  Scores: 
Rho  =  .50 

Figure  4.  Difference  Between  Compensatory  and  Noncompensatory  True  Scores: 
Rho  =  .75 

Figure  5.  Absolute  Difference  Between  Compensatory  and  Noncompensatory  Test 
Information  Vectors:  Rho  =  .00 

Figure  6.  Absolute  Difference  Between  Compensatory  and  Noncompensatory  Test 
Information  Vectors:  Rho  =  .25 

Figure  7.  Absolute  Difference  Between  Compensatory  and  Noncompensatory  Test 
Information  Vectors:  Rho  =  .  50 

Figure  8.  Absolute  Difference  Between  Compensatory  and  Noncompensatory  Test 
Information  Vectors:  Rho  =  .75 
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University  of  Wisconsin 
Department  of  Statistics 
1210  West  Dayton  Street 
Madison.  W1  53705 

Dr.  Michael  Levine 
Educational  Psychology 
210  Education  Bldg. 

University  of  Illinois 
Champaign,  IL  61801 

Dr.  Charles  Lewis 
Educational  Testing  Service 
Pnnceton.  NJ  08541-0001 

Mr.  Rodney  Urn 
University  of  Illinois 
Department  of  Psychology 
603  EL  Daniel  Sl 
Champaign,  IL  61820 

Dr.  Robert  L  Linn 
Campus  Box  249 
University  of  Colorado 
Boulder,  CO  80309-0249 

Dr.  Robert  Lock  man 
Center  for  Naval  Analysis 
4401  Ford  Avenue 
P.O.  Box  16268 
Alexandna.  VA  22302-0268 

Dr.  Fredenc  M.  Lord 
Educational  Testing  Service 
Princeton.  NJ  08541 

Dr.  Richard  Luccht 
ACT 

P.  O.  Box  168 
Iowa  City,  IA  52243 


American  College  Tearing  Program/Rectase 


1 LW/90 


Dr.  George  B.  Macready 
Department  of  Measurement 
Statistics  Sl  Evaluation 
College  of  Education 
University  of  Maryland 
College  Park,  MD  20742 

Dr.  Gary  Marco 
Stop  31-E 

Educational  Testing  Service 
Princeton,  NJ  06451 

Dr.  Qessen  J.  Martin 
Office  of  Chief  of  Naval 
Operations  (OP  13  F) 

Navy  Annex,  Room  2832 
Washington,  DC  20350 

Dr.  James  R.  McBride 
HumRRO 

6430  Elmhurst  Drive 
San  Diego,  CA  92120 

Dr.  Clarence  C  McCormick 
HO,  USMEPCOM/MEPCT 
2500  Green  Bay  Road 
North  Chicago,  IL  60064 

Mr.  Christopher  McCusker 
University  of  Illinois 
Department  of  Psychology 
$03  E.  Daniel  Sl 
Champaign,  IL  61820 

Dr.  Robert  McKinley 
Educational  Testing  Service 
Pr  xeton,  NJ  06541 

Mr.  Alan  Mead 
do  L>r.  Michael  Levine 
Educational  Psychology 
210  Education  Bldg, 

University  of  Illinois 
Champaign,  IL  61801 

Dr.  Timothy  Miller 
ACT 

P.  O.  Box  168 
Iowa  City,  1A  52243 

Dr.  Robert  Miskvy 
Educational  Testing  Service 
Princeton,  NJ  06541 

Dr.  William  Montague 

NPRDC  Code  13 

San  Diego.  CA  92152-6800 

Ms.  Kathleen  Moreno 
Navy  Personnel  R&D  Center 
Code  62 

San  Diego,  CA  92152-6800 

Headquarters  Marine  Corps 
Code  MPI-20 
Washington.  DC  20380 

Dr.  Ratna  Nandakumar 
Educational  Studies 
Willard  HalL  Room  213E 
University  of  Delaware 
Newark.  DE  19716 

Library,  NPRDC 
Code  P201L 

San  Diego.  CA  92152-6800 
Librarian 

Naval  Center  for  Applied  Research 
m  Artificial  Intelligence 
Naval  Research  Laboratory 
Code  5510 

Washington,  DC  20375-5000 


Dr.  Harold  F.  O’Neil,  Jr. 

School  of  Education  -  WPH  801 
Department  of  Educational 
Psychology  Sl  Technology 
University  of  Southern  California 
Los  Angeles,  CA  9006*0031 

Dr.  James  B.  Olsen 
W1CAT  Systems 
1875  South  State  Street 
Orem,  UT  84058 

Office  of  Naval  Research, 

Code  1142CS 
800  N.  Quincy  Street 
Arlington,  VA  22217-5000 
(6  Copies) 

Dr.  Judith  Orasanu 
Basic  Research  Office 
Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

Dr.  Jesse  Odansky 
institute  for  Defense  Analyses 
1601  N.  Beauregard  Sl 
Alexandria,  VA  22311 

Dr.  Peter  J.  Pash  ley 
Educational  Testing  Service 
Rosedaie  Road 
Princeton,  NJ  06541 

Wayne  M.  Patience 
American  Council  on  Education 
GED  Testing  Service,  Suite  20 
One  Dupont  Grde,  N’W 
Washington,  DC  20036 

Dr.  James  Paulson 
Department  of  Psychology 
Port/and  State  Umvenuy 
P.O.  Box  751 
Portland.  OR  97207 

DcpL  of  Administrative  Sciences 
Code  54 

Naval  Postgraduate  School 
Monterey.  CA  93943-5026 

Dr.  Mark  D.  Reckase 
ACT 

P.  O.  Box  168 
I cwa  Gty.  LA  52243 

Dr.  Malcolm  Ree 
AFMRL/MOA 
Brooks  AFB,  TX  78235 

Mr.  Steve  Reiss 
N660  Elliott  Hall 
University  of  Minnesota 
75  E.  River  Road 
Minneapolis,  MN  55455-0344 

Dr.  Carl  Ross 
CNET-PDCD 
Building  90 

Great  Lakes  NTC,  IL  60068 
Dr.  J.  Ryan 

Department  of  Education 
University  of  South  Carolina 
Columbia,  SC  29208 

Dr.  Fumiko  Samejima 
Department  of  Psychology 
University  of  Tennessee 
310B  Austin  Peay  Bldg 
Knoxville.  TN  37916-0900 

Mr.  Drew  Sands 
NPRDC  Code  62 
San  Diego,  CA  92152-6800 


Lowell  Schoer 

Psychological  SL  Quantitative 
Foundations 
College  of  Education 
University  of  Iowa 
Iowa  Gty,  IA  52242 

Dr.  Mary  Schratz 
4100  Partude 
Carlsbad,  CA  92006 

Dr.  Dan  Segall 

Navy  Personnel  RAD  Center 

San  Diego,  CA  92152 

Dr.  Robin  Sbealy 
University  of  Illinois 
Department  of  Statistics 
101  Illini  Hail 
725  South  Wright  Sl 
Champaign,  IL  61820 

Dr.  Kazuo  Sbigemasu 
7-9-24  Kugenuma-Kaigan 
Fujisawa  251 
JAPAN 

Dr.  Randall  Shumaker 
Naval  Research  Laboratory 
Code  5510 

4555  Overtook  Avenue,  S.W. 
Washington,  DC  20375-5000 

Dr.  Richard  E.  Snow 
School  of  Education 
Stanford  University 
Stanford,  CA  94305 

Dr.  Richard  G  Sorensen 
Navy  Personnel  RAD  Center 
San  Diego,  CA  92152-6800 

Dr.  Judy  Spray 

Iowa  Gty,  LA  52243 

Dr.  Martha  Stocking 
Educational  Testing  Service 
Princeton,  NJ  08541 

Dr.  Peter  Stoloff 
Center  for  Naval  Analysis 
4401  Ford  Avenue 
P.O.  Box  162*8 
AJeiondna.  VA  22302-0268 

Dr.  William  Siout 
University  of  'Hmo-s 
Department  of  Statistics 
10!  Illim  Hall 
725  Souih  VSr.ght  Sl 
Champaign.  IL  61820 

Dr.  Hsnharan  Swammathan 
Laboratory  of  Psychometric  and 
Evaluation  Research 
School  of  Education 
University  of  Massachusetts 
Amherst,  MA  01003 

Mr.  Brad  Syrnpson 

Navy  Personnel  RAD  Center 

Code-62 

San  Diego,  CA  92152-6800 

Dr.  John  Tangney 
AFOSR/NL  Bldg  410 
Boiling  AFB.  DC  20332-6448 

Dr.  Kjkumi  Tatsuoka 
Educational  Tcahng  Service 
Mail  Stop  03-T 
Princeton.  NJ  OR541 
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Dr.  Maurice  Tatauokj 
Educational  Testing  Service 
Mail  Slop  0E3-T 
Princeton,  NJ  08541 

Dr.  David  Thissen 
Department  of  Psychology 
University  of  Kansas 
Lawrence,  KS  66044 

Mr.  Thomas  J.  Thomas 
Johns  Hopkins  University 
Department  of  Psychology 
Charles  A  34th  Street 
Baltimore,  MD  21218 

Mr.  Gary  Tbomasson 
University  of  Illinois 
Educational  Psychology 
Champaign.  IL  61820 

Dr.  Robert  Tsutakawa 
University  of  Missouri 
Department  of  Statistics 
222  Math.  Sciences  Bldg, 

Columbia,  MO  65211 

Dr.  Ledyard  Tucker 
University  of  Illinois 
Department  of  Psychology 
603  E.  Daniel  Street 
Champaign,  IL  61820 

Dr.  David  Vale 
Assessment  Systems  Corp. 

2233  University  Avenue 
Suite  440 

SL  Paul,  MN  55114 

Dr.  Frank  L.  Vicino 
Navy  Personnel  RAD  Center 
San  Diego,  CA  92152-4800 

Dr.  Howard  Wainer 
Educational  Testing  Service 
Princeton,  NJ  08541 

Dr.  Michael  T.  Waller 
University  of  Wiscontin-Miiwaukee 
Educational  Psychology  Department 
Bo*  413 

Milwaukee,  W1  53201 

Dr.  Ming-Mei  Wang 
Educational  Testing  Service 
Mail  Stop  03- T 
Princeton,  NJ  08541 

Dr.  Thomas  A.  Warm 
FAA  Academy  AAC934D 
P.O.  Box  25082 
Oklahoma  Gty,  OK  73125 

Dr.  Brian  Waters 

HumRRO 

1100  S.  Washington 

Alexandria,  VA  22314 


Dr,  Rand  R.  Wilcox 
University  of  Southern 
California 

Department  of  Psychology 
Los  Angeles,  CA  90089-1061 

German  Military  Representative 
ATTN:  Wolfgang  Wildgrube 
Streitkraefteamt 
D-5300  Bonn  2 
4000  Brandywine  Street,  NW 
Washington.  DC  20016 

Dr.  Bruce  Williams 
Department  of  Educational 
Piycboiojy 
University  of  Illinois 
Urbans,  IL  61801 

Dr.  Hilda  Wing 

Federal  Aviation  Administration 
800  Independence  Ave.  SW 
Washington,  DC  20591 

Mr.  John  H.  Wolfe 

Navy  Personnel  RAD  Center 

San  Diego,  CA  92152-6800 

Dr.  George  Wong 
Bios ca Lubes  Laboratory 
Memorial  Sloan-Keuering 
Cancer  Center 
1275  York  Avenue 
New  York,  NY  10021 

Dr.  Wallace  Wulfeck.  Ill 
Navy  Personnel  RAD  Center 
Code  51 

San  Diego,  CA  92152-6800 

Dr.  Ken  taro  Yamamoto 
02-T 

Educational  Testing  Service 
Rosedale  Road 
Princeton.  NJ  08541 

Dr.  Wendy  Yen 
CTB/McGraw  Hill 
Del  Monte  Research  Part 
Monterey.  CA  93940 

Dr.  Joseph  L  Young 
National  Science  Foundation 
Room  320 
1800  G  Street,  N.W. 

Washington,  DC  20550 

Mr.  Anthony  R.  Zara 
National  Council  of  State 
Boards  of  Nursing,  Inc 
625  North  Michigan  Avenue 
Suite  1544 
Chicago,  IL  60611 


Dr.  David  J.  Weiss 
N660  EJliou  Hall 
Univetsity  of  Minnesota 
75  E.  River  Road 
Minneapolis,  MN  55455-0344 

Dr.  Ronald  A.  WciUmao 
Bos  146 

Carmel,  CA  93921 

Major  John  Welsh 
AFHRL/MOAN 
Brooks  AFa  TX  78223 

Dr.  Douglas  Wetzel 
Code  51 

Navy  Personnel  RAD  Center 
San  Diego,  CA  92152-6800 


